Prevalence-value-accuracy plots: a new method for comparing diagnostic tests based on misclassification costs.

The clinical accuracy of diagnostic tests commonly is assessed by ROC analysis. ROC plots, however, do not directly incorporate the effect of prevalence or the value of the possible test outcomes on test performance, which are two important factors in the practical utility of a diagnostic test. We describe a new graphical method, referred to as a prevalence-value-accuracy (PVA) plot analysis, which includes, in addition to accuracy, the effect of prevalence and the cost of misclassifications (false positives and false negatives) in the comparison of diagnostic test performance. PVA plots are contour plots that display the minimum cost attributable to misclassifications (z-axis) at various optimum decision thresholds over a range of possible values for prevalence (x-axis) and the unit cost ratio (UCR; y-axis), which is an index of the cost of a false-positive vs a false-negative test result. Another index based on the cost of misclassifications can be derived from PVA plots for the quantitative comparison of test performance. Depending on the region of the PVA plot that is used to calculate the misclassification cost index, it can potentially lead to a different interpretation than the ROC area index on the relative value of different tests. A PVA-threshold plot, which is a variation of a PVA plot, is also described for readily identifying the optimum decision threshold at any given prevalence and UCR. In summary, the advantages of PVA plot analysis are the following: (a) it directly incorporates the effect of prevalence and misclassification costs in the analysis of test performance; (b) it yields a quantitative index based on the costs of misclassifications for comparing diagnostic tests; (c) it provides a way to restrict the comparison of diagnostic test performance to a clinically relevant range of prevalence and UCR; and (d) it can be used to directly identify an optimum decision threshold based on prevalence and misclassification costs.